The present invention relates to displaying images on a display.
The most commonly used method for displaying high-resolution images on a lower resolution color mosaic (i.e. color matrix) display is to prefilter and re-sample the pixels 2 of the high-resolution image 4 down to the resolution of the low-resolution display 6, as shown in FIG. 1. In the process, the R, G, B values of selected color pixels 8 are mapped to the separate R, G, B elements 10, 12 and 14 of each display pixel 16. These R, G, B elements 10, 12 and 14 of a display pixel are sometimes also referred to as subpixels. Because the display device does not allow overlapping color elements the subpixels are spatially distinct. In addition, the subpixels can only take on one of the three R, G, or B colors. The color's amplitude, however, can be varied throughout the entire grey scale (i.e. brightness) range (e.g., 0-255 for “8-bit” pixels). The subpixels often have approximately (neglecting the black regions) a 1:3 aspect ratio (width:height), so that the resulting pixel 16 is square. The aforementioned subsampling/rendering techniques fail to consider the fact that the display's R, G, and B subpixels are spatially displaced; in fact the pixels of the low resolution image are assumed to be overlapping in the same manner as they are in the high-resolution image. This type of sampling may be referred to as subsampling, traditional subsampling, or ordinary subsampling.
The pixels of the high-resolution image 4 are shown as three slightly offset stacked squares 8 to indicate their RGB values are associated with the same spatial position (i.e., pixel), generally referred to as co-sited subpixels. One display pixel 16 on a color mosaic display, consisting of one each of the R, G and B subpixels 10, 12 and 14 is shown as part of the lower-resolution striped display 6 in FIG. 1.
In the example shown in FIG. 1, the high-resolution image has 3× more resolution than the display (in both horizontal and vertical dimensions). In the case that prefiltering is omitted, the subsampling process would cause undesirable aliasing artifacts, and, accordingly, various methods are used, such as averaging the neighboring un-sampled pixels in with the sampled pixel, to reduce the aliasing. In addition, the subsampling technique of FIG. 1 results in mis-registration of the color fields each of which carries a portion of the luminance information. This is due to the displaced subpixels, and leads to a loss of luminance resolution attainable at the subpixel sampling rate.
It is noted that the technique of weighted averaging of neighboring elements while subsampling is mathematically equivalent to FIR prefiltering the high resolution image. Also, it is noted that techniques of selecting a different pixel than the leftmost (as shown in FIG. 1) can be considered as a prefiltering that affects only the phase spectrum (spatial shift) of the high resolution image. Thus, most of the processing associated with reducing aliasing may be viewed as a filtering operation on the high-resolution image, even if the filter kernel is applied only at the sampled pixel positions.
It has been realized that the aforementioned techniques do not take advantage of potential display resolution beyond that determined by the Nyquist limit due to the display pixel spacing. Information regarding potential display resolution is discussed by R. Fiegenblatt (1989), “Full color imaging on amplitude color mosaic displays” Proc. SPIE V. 1075, 199-205; and J. Kranz and L. Silverstein (1990) “Color matrix display image quality: The effects of luminance and spatial sampling,” SID Symp. Digest 29-32, incorporated herein by reference.
For example, in the display shown in FIG. 1, while the display pixel 16 resolution is ⅓ that of the pixel resolution of the high resolution image (source image) 4, the subpixels 10, 12 and 14 of the low resolution display are at a resolution equal to that of the high resolution image (in the horizontal dimension). This may be taken advantage of as shown in FIG. 2. For each pixel of the high resolution image a luminance value exists, a portion of which is mapped to the corresponding subpixel of the low resolution image. In this manner, a portion of the high resolution luminance in the image 4 is preserved in the subpixels of the low resolution image shown on the display. In this manner, the luminance resolution of the displayed image is enhanced. This approach is shown in FIG. 2, where the R, G, and B subpixels 10, 12 and 14 of the low resolution display are taken from the corresponding colors of different pixels 11, 13 and 15 of the high-resolution image. The subsampling acts to re-register the color fields of the display which serves to increase the available luminance bandwidth. Luminance resolution is increased when the RGB pre-filter bandwidths are adjusted to make use of this bandwidth. Sampling which comprises mapping color elements from different high-resolution image pixels to the subpixels of a display pixel may be referred to as “subpixel subsampling”. Unfortunately, this “naïve” method of subpixel subsampling produces color artifacts (a.k.a., “color aliasing” or “chrominance aliasing”) in addition to the beneficial enhanced luminance resolution. This color aliasing may be viewed, mathematically, as carrying the additional luminance resolution. To effectively reduce the color aliasing seen by the eye, while retaining a substantial part of the gained luminance resolution, various properties of the Human Visual System may be used. In FIG. 3, luminance Contrast Sensitivity Function (CSF) 17 refers to the luminance sensitivity of the human visual system with respect to spatial frequency. The chrominance CSF 19 refers to the chromatic sensitivity of the human visual system with respect to spatial frequency. The human visual system processes chromatic content as isoluminant modulations between red and green, and between blue and yellow. The color difference signals R-Y and B-Y of typical video are rough approximations to these modulations. For most observers, the bandwidth of the chromatic frequency response is ½ that of the luminance frequency response. Sometimes, the bandwidth of the blue-yellow modulation response is even less, down to about ⅓ of the luminance. It may be observed that viewers can not effectively see high frequency color content.
The visual frequency responses (CSFs) shown in FIG. 3 are idealized. In practice, they have a finite falloff slope, more representatively shown by the curve in FIG. 5A marked 30 (luminance CSF), and 32, 34 (chrominance CSFs). It is shown as the solid line 30 that has a cutoff frequency near 1.5 cy/pixel (display pixel), and is bandpass in shape with a peak near 0.2 cy/pixel. The R:G CSF 32 is shown as the dashed line that is lowpass with a cutoff frequency near 0.5 cy/pixel. The B:Y CSF 34 is shown as the long dashed low pass curve with a cutoff frequency similar to the R:G CSF, but with lower peak response. The range between the cutoff frequencies of the chromance CSF 32 and 34 and the luminance CSF 30 is the region where one may allow chromatic aliasing in order to improve luminance resolution. The chromatic aliasing will not be visible to the human eye because it falls outside the chromance CSF.
FIG. 5A also shows an idealized image power spectra 36 as a 1/f function, appearing in the figure as a straight line with a slope of −1 (since the figure is using log-log axes). This spectrum will repeat at the sampling frequency. The repeat spectrum 38 is due to the pixel sampling rate, and the repeat spectrum 40 is due to the subpixel sampling rate. Note that the shapes of the repeat spectra are different than the 1/f base band spectra 36 because they are plotted on log-log axes. The portions of these repeat spectra 38 and 40 that extend below their respective Nyquist frequencies represent aliasing. The leftmost spectrum 38, contributes to the chromatic aliasing since it is due to the pixel sampling rate. The right hand spectrum 40 contributes to the luminance aliasing. However, for the 1/f power spectrum shown, the luminance aliasing is nearly zero.
In FIG. 5A, no prefiltering has been applied to the source spectra. Consequently, aliasing, due to the pixel sampling (i.e., chromatic aliasing), extends to very low frequencies 35. Thus color artifacts will, in general, be visible (depending on the noise and contrast of the display). The chrominance aliasing is visible because it extends into the chromance CSF.
Referring to FIG. 4, curve 22 illustrates the frequency response of a filter that is one display pixel wide. In FIG. 5B, a prefilter was applied (a rect function in the spatial domain equal to three source image pixels, i.e., one display pixel), shown in FIG. 4 as a dashed-dotted line 22, to the source power spectrum, and it affects the baseband spectrum 42 in the region of 0.5 cy/pixel and greater, causing it to have a slope steeper than −1 shown at 44. The steeper slope effectively reduces the effects of the chromatic aliasing. The repeat spectra 38a and 40a also show the effect of this prefilter. For example, the tail 35 (FIG. 5A) is dramatically reduced to tail 46 (FIG. 5B) due to this filter. In particular, the visible chromatic aliasing, that is, aliasing under the two chrominance CSFs 32a and 34a, is reduced. However, it can be observed that this prefiltering also removes a substantial portion of the luminance resolution that is desirable to keep.
Accordingly, with such a prefiltering technique, desirable luminance resolution is sacrificed in order to reduce the undesirable chrominance aliasing.